- 75 - 



WHAT IS CLAIMED IS: 



1. A three-dinJ^nsional position and orientation 

\ . . 

sensing apparatus comprising: 

image input meansXfor inputting an image acquired 

5 by an image acquisition ^apparatus and having at least 

\ 

three markers, three-dimensional positional information 
of which with respect to aiis^ object to be measured is 
known in advance; 

region extracting means for extracting a region 

10 corresponding to each marker o\^the image; 

marker identifying means for identifying the 
individual markers from the characteristics of the 
appearance of the markers in the extracted regions; and 
position and orientation calculating means for 

15 calculating the three-dimensional position and 

orientation of the object to be measured^ with respect 
to the image acquisition apparatus, by us^ng positions 
of the identified markers on the image, and the three- 
dimensional positional information of the markers with 

20 respect to the object to be measured. 

2. A three-dimensional position and orientation 
sensing apparatus comprising: 

image input means for inputting an image acquired 
by an image acquisition apparatus and having at l^ast 
25 four markers, three-dimensional positional information 

is\ 



of which with respect to an object to be measured 
known in advance; 



\ 



-It. 
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region extracting means for extracting an area 
corresponding to each marker on the image; 

marker identifying means for identifying the 
individual markers^from the characteristics of the 
5 appearance of the markers in the extracted regions; 

marker selecting\means for selecting three markers 
from the markers identi^fied by the marker identifying 
means ; 

parameter calculating^eans for calculating a 
10 plurality of sets of parameters for calculating the 

three-dimensional position a\d^ orientation of the 
object to be measured with respect to the image 
acquisition apparatus, by utilizJLng positions on the 
image of the three markers selected by the marker 
15 selecting means and three-dimensional positional 

information of each marker with respect to the object 
to be measured; and 

parameter selecting means for selecting one set of 
parameters by evaluating application of the parameters 

\ 

20 calculated by the parameter calculating means, to 

markers not selected by the marker selecting means. 

\ 

3. A three-dimensional position and orientation 

\ 

sensing apparatus according to claim 2, further 

\ 

comprising parameter improving means for improving one 

\ 

25 set of parameters selected by the parameter select:ing 

means, by evaluating the set of parameters through^ 
application thereto of three-dimensional positional 
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information of each marker with respect to the object 
to be measured. 

4. A three-din^ensional position and orientation 
sensing apparatus comprising: 
5 image input means \f or inputting an image acquired 

by an image acquisition apparatus and having at least 
three markers, three-dimensional positional information 
of which with respect to any object to be measured is 
known in advance; 

10 region extracting means for extracting a region 

corresponding to each marker on\the image; 

marker identifying means foi^ identifying the 
individual markers from the characs^eristics of the 
appearance of the markers in the extracted regions; 

15 distance estimating means for estimating a 

distance to each marker based on the sUze of the 
identified marker in the image; and 

position and orientation calculating means for 
calculating the three-dimensional position^and 

2 0 orientation of the object with respect to the image 

acquisition apparatus, by using the estimated\^ distance 
to each marker, a position of the marker on th^e^ image 
and three-dimensional positional information of "^^his 
marker with respect to the object, \ 

25 5. A three-dimensional position and orientation 

sensing apparatus comprising: ^ 

image input means for inputting an image acquirecl 

\ 

\ 
\ 



# 
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by an image acquisition apparatus and having markers, 
three-dimensional\positional information of which with 
respect to an objec^ to be measured is known in 
advance; \ 
5 image reducing means for reducing the size of the 

input image ; 

region extracting m^ans for extracting a region 
corresponding to each marker on the reduced image; and 
position and orientation calculating means for 
10 calculating the three-dimens\Lonal position and 

orientation of the object with respect to the image 

of each marker on the image and three-dimensional 
positional information of each marker with respect to 
15 the object. 

6. A three-dimensional position^ and orientation 
sensing apparatus according to claim 4^, further 
comprising : 

feature extracting means for extracting features 
20 of the object to be measured, from the ima^e; and 

means for calibrating the position and^ orientation 
of the calculated object, from the position \)f the 
extracted features . \ 

\ 

7 . A three-dimensional position and orieril:ation 

\ 

25 sensing apparatus according to claim 5, further \ 



acquisition apparatus, by using^ the extracted position 



comprising: 

\ 

feature extracting means for extracting features 
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of the object to be measured, from the image; and 

means for ca\l.ibrating the position and orientation 
of the calculated object, from the position of the 
extracted .eatu.es y he i.a,e. 
5 8. A three-dimensional position and orientation 

sensing method for measiiring the position and 
orientation of an objectX to be measured with respect to 
an image acquisition apparatus, by analyzing an image 
acquired by this image acqu^isition apparatus, the 
10 method comprising the steps of: 

inputting an image acquirN^d by an image 
acquisition apparatus and having at least three markers, 
three-dimensional positional inf^xmation of which with 
respect to an object to be measured is known in 
15 advance; \ 

extracting a region corresponding to each marker 
on the image; \ 

identifying the individual markers from the 
characteristics of the appearance of the markers in the 
20 extracted regions; and 

calculating the three-dimensional position and 
orientation of the object to be measured with respect 
to the image acquisition apparatus, by using positions 
of the identified markers in the image, and the three- 
25 dimensional positional information of the markers with 

respect to the object to be measured, 

9. A three-dimensional position and orientation 
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sensing methodXfor measuring a position and orientation 
of an object to \)e measured with respect to an image 
acquisition apparatus, by analyzing an image acquired 
by this image acquisition apparatus, the method 
5 comprising the step^\of: 

inputting an image acquired by an image 
acquisition apparatus \nd having at least four markers, 
three-dimensional positional information of which with 
respect to an object to be^ measured is known in 
10 advance; 

extracting a region corresponding to each marker 
on the image; 

identifying the individualX markers from the 
characteristics of the appearance^ of the markers in the 
15 extracted regions; 

selecting three markers from the markers 
identified by the marker identif ying^means ; 

calculating a plurality of sets ^f^ parameters for 

calculating the three-dimensional position and 

\ 

20 orientation of the object to be measured with respect 

\ 

to the image acquisition apparatus, by utilizing 
positions in the image of the three markers selected by 
the marker selecting means and three-dimensio\ial 
positional information of each marker with respect to 
25 the object; and \ 

selecting one set of parameters by evaluating 
application of the parameters calculated by the 
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parameter calcinating means, to markers not selected by 
the marker selecting means. 

10. A three-aimensional position and orientation 
sensing method according to claim 9, further comprising 
5 the step of improvino one set of parameters selected by 

the parameter selecting means, by evaluating the set of 
parameters through appiication thereto of three- 
dimensional positional information of each marker with 

• ^ 

respect to the object. 
10 11. A three-dimensional position and orientation 

•2 \ 

sensing method for measuring^ the position and 
W orientation of an object to be measured with respect to 

\ 

O an image acquisition apparatus ,\ by analyzing an image 

p acquired by this image acquisition apparatus, the 

15 method comprising the steps of: 

''^ inputting an image acquired by. an image 

acquisition apparatus and having atXleast three markers, 
three-dimensional positional information of which with 
respect to an object to be measured is^known in 
2 0 advance; 

extracting a region corresponding to\each marker 
on the image; 

identifying the individual markers fromXthe 
characteristics of the appearance of the markers in the 
2 5 extracted areas; 

estimating the distance to each marker based on a 
size of the identified marker in the image; and 
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calculating the three-dimensional position and 
orientation of mie object to be measured with respect 
to the image acqirisition apparatus, by using the 
estimated distanceXto each marker, a position of the 
5 marker on the imageXand three-dimensional positional 

information of this marker with respect to the object. 

12. A three-dimer^sional position and orientation 
sensing method for measuring the position and 
orientation of an objectVto be measured with respect to 

10 an image acquisition apparatus, by analyzing an image 

acquired by this image acqu^isition apparatus, the 
method comprising the steps of: 

inputting an image acquired by an image 
acquisition apparatus and having markers, three- 
15 dimensional positional information of which with 

respect to an object to be measured is known in 
advance ; 

reducing the input image; 

extracting a region corresponding to each marker 
20 on the reduced image; and \ 

calculating the three-dimensional position and 

\ 

\ 

orientation of the object to be measured with respect 
to the image acquisition apparatus, by usi\g the 
extracted position of each marker on the image and 
25 three-dimensional positional information of each marker 

with respect to the object to be measured. 

13. A three-dimensional position and orientation 

\ 
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sensing method according to claim 11, further 
comprising the s^eps of: 

\ 

extracting features of the object to be measured, 
from the image; and\ 
5 calibrating theXposition and orientation of the 

calculated object, frojn a position of the extracted 
features in the image, 

14. A three-dimensi^onal position and orientation 

\ 

sensing method according to claim 12, further 
10 comprising the steps of: 

extracting features of\the object to be measured, 
from the image; and 

calibrating the position ^and orientation of the 
calculated object, from positions of the extracted 
15 features in the image. 

15. An article of manufacture comprising a 
computer-readable recording medium \having computer- 
readable program coding means as a processing program 
recorded for measuring the position aiid orientation of 

20 an object to be measured with respect to an image 

acquisition apparatus, by analyzing by computer an 
image acquired by this image acquisition^apparatus , the 
computer-readable program coding means comprising: 

computer-readable programming means for making an 

25 image to be input, the image having been acquired by an 

image acquisition apparatus and having at leas^ three 
markers, three-dimensional positional information of 
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which with Vrespect to an object to be measured is known 
in advance; 

computerWeadable programming means for making an 
area corresponasj.ng to each marker on the image to be 
5 extracted; 

computer-readable programming means for making the 
individual markers to be identified from the 
characteristics of the appearance of the markers in the 
extracted regions ; and\ 
10 computer-readable programming means for making the 

three-dimensional position and orientation of the 
object to be measured withXrespect to the image 
acquisition apparatus to be calculated, by using 
positions of the identified markers on the image, and 
15 the three-dimensional positionai^l information of the 

\ 

markers with respect to the object to be measured. 



16 . An article of manuf acture»^comprising a 

computer-readable recording medium having computer- 

\ 

readable program coding means as a processing program 

\ 

20 recorded for measuring the position and orientation of 

an object to be measured with respect to an image 

\ 

acquisition apparatus, by analyzing by computer an 



image acquired by this image acquisition apparatus, the 

computer-readable program coding means compr^ising: 

\ 

25 computer-readable programming means for fnaking an 

image to be input, the image having been acquired by an 

\ 

image acquisition apparatus and having at least f.our 

\ 
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markers, three-dimensional positional information of 
which with respect^ to an object to be measured is known 
in advance; 

computer-readable programming means for making a 
5 region corresponding to each marker on the image to be 

extracted, making the iVdividual markers to be 
identified from the characteristics of the appearance 
of the markers in the extracted regions, and making 
three markers to be selected from the identified 
10 markers; 

computer-readable programming means for making a 
plurality of sets of parameter^^or calculating the 
three-dimensional position and orientation of the 
object to be measured with respecA to the image 
15 acquisition apparatus to be calculated, by using 

\ 

positions of the selected three markers on the image, 
and the three-dimensional positional information of the 

\ 

markers with respect to the object to be measured; and 
computer-readable programming meansX^or making one 
2 0 set of parameters selected, by evaluating the non- 

selected markers through application of the\calculated 
parameters thereto . 



17 . An article of manufacture comprising a 
computer-readable recording medium storing a thiree- 
25 dimensional position and orientation sensing processing 

program according to claim 16, wherein 

the computer-readable program coding means as a 
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processing program further comprises: 

computer \:eadable programming means for making one 
set of parameteiis selected by the parameter selecting 
means to be improved, by evaluating the set of 
5 parameters throughN^pplication thereto of three- 

dimensional positional infoirmation of each marker with 
respect to the object A 

18- An article of manufacture comprising a 
computer-readable recording medium having computer- 
10 readable program coding means as a processing program 

recorded for measuring the position and orientation of 
an object to be measured with\respect to an image 
acquisition apparatus, by analyzing by computer an 
image acquired by this image acqudsition apparatus, the 
15 computer-readable program coding me^ns comprising: 

computer-readable programming means for making an 
image to be input, the image having be^n acquired by an 
image acquisition apparatus and having at least three 
markers, three-dimensional positional information of 

2 0 which with respect to an object to be measured is known 

\ 

in advance; 

computer-readable programming means for making a 

\ 

region corresponding to each marker on the image to be 
extracted; 

25 computer-readable programming means for makirig the 

individual markers to be identified from the charac\^ 
teristics of the appearance of the markers in the \ 

\ 

\ 



\ 
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extracted regions ; 

computer-\readable programming means for making a 
distance to each marker to be estimated based on the 
size of the identiified marker on the image; and 
5 computer-readable programming means for making the 

three-dimensional position and orientation of the 
object to be measured^ with respect to the image 
acquisition apparatus ^ be calculated, by using the 
estimated distance to each marker, a position of the 
10 marker on the image and t^hree-dimensional positional 

information of this marker\with respect to the object 
to be measured. 

19. An article of manufacture comprising a 
computer-readable recording medium having computer- 
15 readable program coding means as a processing program 

recorded for measuring the positron and orientation of 
an object to be measured with respect to an image 
acquisition apparatus, by analyzingVby computer an 
image acquired by this image acquisition apparatus, the 
2 0 computer-readable program coding means\comprising: 

computer-readable programming means^for making an 
image to be input, the image having been acquired by an 
image acquisition apparatus and having markers, three- 
dimensional positional information of which Vith 
2 5 respect to an object to be measured is known in 

advance; 

computer-readable programming means for maJoing the 

\ 
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input image be reduced; 

computer-readable programming means for making a 
region corresponding to each marker to be extracted on 
the reduced imaged and 
5 computer-readable programming means for making the 

three-dimensional poWtion and orientation of the 
object to be measuredXwith respect to the image 
acquisition apparatus to be calculated, by using the 
extracted position of eac^h marker on the image and 
10 three-dimensional positional information of each marker 

with respect to the object 

20. An article of manufacture comprising a 
computer-readable recording medium storing a three- 
dimensional position and orientation sensing processing 

15 program according to claim 18, wherein 

the computer-readable program coding means as a 
processing program further comprises: 

\ 

computer-readable programming means for making 
features of the object to be measured A to be extracted 
2 0 from the image, and making the posit ion\ and orientation 

of the calculated object to be calibrated^, from a 
position of the extracted features 

21. An article of manufacture comprising a 
computer-readable recording medium storing a\three- 

25 dimensional position and orientation sensing processing 

program according to claim 19, wherein 

the computer-readable program coding means ds a 
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0^ 



V 



processing program ^rther comprises: 

computer-readabrfe programming means for making 
features of the object Nto be measured, to be extracted 
from the image, and makirig the position and orientation 
of the calculated object tp be calibrated, from a 
position of the extracted futures in the image, 

22. Markers having identification marks disposed 
on their planesV wherein 

the externa\ shapes of the identification marks 
10 are circular. 

23. Markers haAfng identification marks disposed 
on their planes, 

each identi.f'iSatidp'^mark having 

a background with a\6^ircular external shape, and 
15 a plurality of ^i^ede^rmined patterns disposed 

inside this background, whei 

it is possibleJito—i'dentiNf y each marker based on a 
combination of colors of the plurality of predetermined 
patterns . 

20 24. Markers having identif icdsjiion marks disposed 

on their planes, 

each identification mark having 
a background with a circular external shape, 
a center mark disposed at the centevp of the 
25 background, and 

a plurality of peripheral marks dispos'^ed at equal 
intervals on a concentric circle encircling tshe center 



4 
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10 



15 



20 




25 



mark, whereiVj 

it is possible to identify each marker based on a 
combination of (colors of the center mark and the 
peripheral marks N 

25. Markers helping identification marks disposed 
on their planes, 

each identif ic^af^pn mark") having 

a backgr^ound with ^ ci^c^flar external shape, and 
a plurality of vcongfektric circled with different 
radiuses disposed insVde tnte backg^und, wherein 

it is possible tc^^id^nt'My each marker based on a 
combination of colors of areas\between the concentric 
circles , 

26. A probe to be used for me'^suring a position, 
the probe comprising: 

a contacting portion as a member\for contacting an 
object to be measured; and 

a mark portion having identificatioiiv marks for 
identifying the probe disposed on the planfe of the mark. 

27. An apparatus ^f or measuring a relative three- 
dimensional position and\ orientation of an object with 



respect to an image acquisition apparatus by measuring 
image positions of a plurajNity of markers mounted on 
the object or near the objecty based on an image of the 
markers acquired by the image Requisition apparatus, 
wherein three-dimensional positrons of the markers are 
known in advance, and the measuring apparatus 
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comprising : 

means for extracting a region corresponding to 
each of the plurality of markers from within the image 
of the markerte ; 
5 means f orVidentif ying the individual markers by 

calculating geomWric characteristics of the region 
corresponding to e^ch of the markers within the image; 
and 

means for estimc^ting the relative three- 
10 dimensional position and orientation of the object with 

respect to the image acquisition apparatus, based on 
the three-dimensional pos\ytions of the markers. 

28. A three-dimensional position and orientation 

\ 

sensing apparatus according to claim 27, wherein coded 
15 patterns are disposed within th^ individual markers . 

29. A three-dimensional position and orientation 

\ 

sensing apparatus according to cla^^m 27, wherein 

the means for identifying the individual markers 
by calculating geometric characterist^ics of the region 

2 0 of each marker within the image measure^ the size of 

the region of each marker within the image, and 

the means for estimating the relative, three- 
dimensional position and orientation of theXobject with 
respect to the image acquisition apparatus calculates 

25 an initial estimate of the distance between the image 

acquisition apparatus and a marker based on the \ 

\ 

measured value of the size of the marker region, and 

\ 



• # 
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calculates the relative thr^e-dimensional position and 
orientation of the object wit!i respect to the image 
acquisition apparatus based onVhe initial estimate. 

30. A threeAlimensional position and orientation 
5 sensing apparatus according to claim 27, wherein the 

image acquisition apparatus is mounted on other 
apparatus, and when tne three-dimensional position and 
orientation relationsha^p between the image acquisition 
apparatus and the other ^^paratus is known in advance 

10 or can be measured s^^^ra^^ely\^^^ three-dimensional 

position and orientat:^cm sensing apparatus measures the 
three-dimensional posit:L^n and orientation of the 
object with respect to the other apparatus by utilizing 
the relative three-dimensional position and orientation 

15 of the object with respect to the^ image acquisition 

apparatus . 

31. A three-dikensional position and orientation 
sensing apparatus according to claim 29, wherein an 
estimate value is calciilated by utilizing only a marker 

20 that takes a range which\makes it possible to estimate 

the relative distance, 

32. A three-dimensional^position and orientation 
sensing apparatus according to claim 29, wherein an 
estimate error variance is calculated for the estimated 

25 relative distance, and the three-dimensional position , 

and orientation of the object withXrespect to the image 

\ 

acquisition apparatus is estimated by taking both the 



\ 



estimate value and the error variance into 
consideration. \^ 

33. A three-dimensional position and orientation 

sensing apparatus according to claim 29, wherein the 

markers are circularX markers , a projected image of each 

marker within the imaW is approximated to an ellipse, 

and the distance from the image acquisition apparatus 

to each marker is estimated by utilizing the length of 

the long axis of the ellipse. 

\ 

34. A three-dimensional position and orientation 

\ 

sensing apparatus according \to claim 33, wherein a 

circular marker is utilized as each marker. 

\^ 

35. A three-dimensional position and orientation 

\ 

sensing apparatus according to claim 28, wherein a 
circular marker is utilized as each marker, and small 
patterns of different colors are used as a code within 
the marker. 

36. A three-dimensional positiori\^and orientation 
sensing apparatus according to claim 28, wherein 
patterns of different colors are generated on a 
concentric circle as the maker. \ 

37. A three-dimensional position and orientation 
sensing apparatus according to claim 33, wherein a 

three-dimensional positional sensing is carried out by 

\ 

adding direction information of the circular marker. 

\ 

38. A three-dimensional position and orientation 

'"^^ 

sensing apparatus according to claim 27, wherein,\ for 
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extracting a marker region from an original image, the 
apparatus generates a reduced image of the original 
image, extracts A candidate region that is considered 
to correspond to aXmarker from the reduced image, 
5 calculates the area\pf the candidate region within the 

original image, and Imen extracts and recognizes a 
region corresponding to the marker within the original 
image • 

39. A three-dimensional position and orientation 
10 sensing apparatus according to claim 27, wherein, after 

a marker having unique geom^^^ric characteristics has 
been recognized, the apparatus extracts from within the 
image a separate marker havingXno unique geometric 
characteristics, by utilizing the two-dimensional or 

15 three-dimensional positional relationship of the first 

marker, and updates the three-dimensional position and 
orientation parameter of the object with respect to the 
image acquisition apparatus, by utilizing the three- 
positional position of the marker with\respect to the 

20 image acquisition apparatus 

40. A three-dimensional position andXorientation 
sensing apparatus according to claim 27, wherein the 
image acquisition apparatus acquires a plurality of 
images and utilizes the plurality of images. \ 

25 41. A three-dimensional position and orierttation 

\ 

sensing apparatus according to claim 27, whereinX 

\ 

markers are mounted on the plane of a sensor prober, and 



\ 
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the sensor probe is used as\an object to be measured, 
whereby to estimate the relat\ve position and 
orientation of the sensor probey^ and to measure the 
position of a probe tip of the s^sor probe. 

42. A three-dimeAsional position and orientation 
sensing apparatus accor^ding to claim 29, wherein the 
shapes of the markersi^re regular polygons 




